109 research outputs found
Applications of correlation inequalities to low density graphical codes
This contribution is based on the contents of a talk delivered at the
Next-SigmaPhi conference held in Crete in August 2005. It is adressed to an
audience of physicists with diverse horizons and does not assume any background
in communications theory. Capacity approaching error correcting codes for
channel communication known as Low Density Parity Check (LDPC) codes have
attracted considerable attention from coding theorists in the last decade.
Surprisingly strong connections with the theory of diluted spin glasses have
been discovered. In this work we elucidate one new connection, namely that a
class of correlation inequalities valid for gaussian spin glasses can be
applied to the theoretical analysis of LDPC codes. This allows for a rigorous
comparison between the so called (optimal) maximum a posteriori and the
computationaly efficient belief propagation decoders. The main ideas of the
proofs are explained and we refer to recent works for the more lengthy
technical details.Comment: 11 pages, 3 figure
Sharp Bounds for Optimal Decoding of Low Density Parity Check Codes
Consider communication over a binary-input memoryless output-symmetric
channel with low density parity check (LDPC) codes and maximum a posteriori
(MAP) decoding. The replica method of spin glass theory allows to conjecture an
analytic formula for the average input-output conditional entropy per bit in
the infinite block length limit. Montanari proved a lower bound for this
entropy, in the case of LDPC ensembles with convex check degree polynomial,
which matches the replica formula. Here we extend this lower bound to any
irregular LDPC ensemble. The new feature of our work is an analysis of the
second derivative of the conditional input-output entropy with respect to
noise. A close relation arises between this second derivative and correlation
or mutual information of codebits. This allows us to extend the realm of the
interpolation method, in particular we show how channel symmetry allows to
control the fluctuations of the overlap parameters.Comment: 40 Pages, Submitted to IEEE Transactions on Information Theor
The adaptive interpolation method for proving replica formulas. Applications to the Curie-Weiss and Wigner spike models
In this contribution we give a pedagogic introduction to the newly introduced
adaptive interpolation method to prove in a simple and unified way replica
formulas for Bayesian optimal inference problems. Many aspects of this method
can already be explained at the level of the simple Curie-Weiss spin system.
This provides a new method of solution for this model which does not appear to
be known. We then generalize this analysis to a paradigmatic inference problem,
namely rank-one matrix estimation, also refered to as the Wigner spike model in
statistics. We give many pointers to the recent literature where the method has
been succesfully applied
Polymer Expansions for Cycle LDPC Codes
We prove that the Bethe expression for the conditional input-output entropy
of cycle LDPC codes on binary symmetric channels above the MAP threshold is
exact in the large block length limit. The analysis relies on methods from
statistical physics. The finite size corrections to the Bethe expression are
expressed through a polymer expansion which is controlled thanks to expander
and counting arguments
The Velocity of the Propagating Wave for General Coupled Scalar Systems
We consider spatially coupled systems governed by a set of scalar density
evolution equations. Such equations track the behavior of message-passing
algorithms used, for example, in coding, sparse sensing, or
constraint-satisfaction problems. Assuming that the "profile" describing the
average state of the algorithm exhibits a solitonic wave-like behavior after
initial transient iterations, we derive a formula for the propagation velocity
of the wave. We illustrate the formula with two applications, namely
Generalized LDPC codes and compressive sensing.Comment: 5 pages, 5 figures, submitted to the Information Theory Workshop
(ITW) 2016 in Cambridge, U
Approaching the Rate-Distortion Limit with Spatial Coupling, Belief propagation and Decimation
We investigate an encoding scheme for lossy compression of a binary symmetric
source based on simple spatially coupled Low-Density Generator-Matrix codes.
The degree of the check nodes is regular and the one of code-bits is Poisson
distributed with an average depending on the compression rate. The performance
of a low complexity Belief Propagation Guided Decimation algorithm is
excellent. The algorithmic rate-distortion curve approaches the optimal curve
of the ensemble as the width of the coupling window grows. Moreover, as the
check degree grows both curves approach the ultimate Shannon rate-distortion
limit. The Belief Propagation Guided Decimation encoder is based on the
posterior measure of a binary symmetric test-channel. This measure can be
interpreted as a random Gibbs measure at a "temperature" directly related to
the "noise level of the test-channel". We investigate the links between the
algorithmic performance of the Belief Propagation Guided Decimation encoder and
the phase diagram of this Gibbs measure. The phase diagram is investigated
thanks to the cavity method of spin glass theory which predicts a number of
phase transition thresholds. In particular the dynamical and condensation
"phase transition temperatures" (equivalently test-channel noise thresholds)
are computed. We observe that: (i) the dynamical temperature of the spatially
coupled construction saturates towards the condensation temperature; (ii) for
large degrees the condensation temperature approaches the temperature (i.e.
noise level) related to the information theoretic Shannon test-channel noise
parameter of rate-distortion theory. This provides heuristic insight into the
excellent performance of the Belief Propagation Guided Decimation algorithm.
The paper contains an introduction to the cavity method
- …